Issue #24 describes the need to adjust the caching of data products. The chosen adjustment is described at #24 (comment) and explained here.

Caching performed by the framework graph

The Phlex framework implements caching to ensure that algorithms can operate on data products from correctly-associated data cells. The caching technology is based largely on oneTBB Flow Graph's tbb::flow::join_node<...> template. This join node is used in tag-matching mode, where the input data on the join node's ports are emitted together when the "tags" corresponding to the data across the input ports match.

To ensure that data from different data layers can be matched together, each data product (or, more properly, the product store that contains the data product) is included as part of a message that contains an ID. The message ID (an object of type std::size_t) is inspected by the tbb::flow::join_node<...> to match data from the join node's ports.

With the current implementation of oneTBB's Flow Graph, there is no way to tell a join node to (e.g.) reuse a particular data product from a run for a given set of spills within that run. The same data product must be repeatedly presented to the join node even though there may be many spills for one run. How this problem is solved depends on the implementation and is described below.

Current implementation

A workflow that demonstrates how this works with the present implementation is shown below.

The nodes provide(A), transform(A), and observe(C) should execute only once per r encountered. The provide(B), transform(B), and transform(C, D) nodes should execute once per (r, s) encountered.

In order to ensure that the transform(C, D) can execute once per spill, the data product $c_r$ must be presented to the join node the same number of times (and with the same message ID) as each of the data products $d_{rs}$ coming from transform(B). This is achieved by sending all of the spills to the provide(A) node, reusing any data products already created by that node, and then passing them downstream until the data product $c_r$ can be re-presented to the join node. This is awkward for various reasons:

1. The nodes provide(A), transform(A), and observe(C) do not conceptually operate on spill-level data products.
2. All nodes must cache the result of the nodes even though that may not be necessary
3. Only the $[c_r] \times [n_{rs}]$ data products are matched with $[d_{rs}]$. The join node has another $[c_r]$ data products cached at one of its input ports until the end of the program (there will be no corresponding $[d_r]$ matches).
4. Some node's caches are flushed prematurely, thus unnecessarily requiring the re-execution a node.

These combined behaviors result in unpredictable results for the user.

With this PR

This PR localizes caching to only the nodes that require multiple inputs.

The benefits to this approach:

1. The nodes provide(A), transform(A), and observe(C) operate only on run-level data products.
2. Only nodes requiring multiple data products as input must cache the required data.
3. There are no unmatched data products at any of the join node's input ports
4. Caches are flushed only when the node that requires a particular data product no longer needs it

The major changes with this PR

1. The multiplexer ("router" in earlier diagrams) has been replaced by an index_router class (labeled "Index Router" in the above image)
2. Explicit "index set" nodes have been added, which are responsible for reacting to data-cell indices belonging to a specific layer (this cleans up an infelicity in how provider nodes were implemented)
3. Caching has been removed from all nodes except any nodes that require joining two or more arguments:
   1. For nodes with two or more products that belong to the same data layer, a multilayer_join_node is created in such a manner that only the standard tbb::join_node<...> is used to tag-match the inputs
   2. For nodes with two or more products that belong to different data layers, a multilayer_join_node is created with "repeaters", which cache data products from the different layers while emitting them as needed to match the other inputs for the node.
4. A one-slot std::tuple<message> message type is no longer used for nodes that consume only one data product, thus removing unnecessary complexity and operations in presenting the data to nodes.